The value of real world evidence: The case of medical cannabis.

Randomised controlled trials (RCTs) have long been considered the gold standard of medical evidence. In relation to cannabis based medicinal products (CBMPs), this focus on RCTs has led to very restrictive guidelines in the UK, which are limiting patient access. There is general agreement that RCT evidence in relation to CBPMs is insufficient at present. As well as commercial reasons, a major problem is that RCTs do not lend themselves well to the study of whole plant medicines. One solution to this challenge is the use of real world evidence (RWE) with patient reported outcomes (PROs) to widen the evidence base. Such data increasingly highlights the positive impact medical cannabis can have on patients' lives. This paper outlines the value of this approach which involves the study of interventions and patients longitudinally under medical care. In relation to CBMPs, RWE has a broad range of advantages. These include the study of larger groups of patients, the use of a broader range and ratio of components of CBMPs, and the inclusion of more and rarer medical conditions. Importantly, and in contrast to RCTs, patients with significant comorbidities-and from a wider demographic profile-can also be studied, so providing higher ecological validity and increasing patient numbers, whilst offering significant cost savings. We conclude by outlining 12 key recommendations of the value of RWE in relation to medical cannabis. We hope that this paper will help policymakers and prescribers understand the importance of RWE in relation to medical cannabis and help them develop approaches to overcome the current situation which is detrimental to patients.

A Multicriteria Decision Analysis Comparing Pharmacotherapy for Chronic Neuropathic Pain, Including Cannabinoids and Cannabis-Based Medical Products.

Background: Pharmacological management of chronic neuropathic pain (CNP) still represents a major clinical challenge. Collective harnessing of both the scientific evidence base and clinical experience (of clinicians and patients) can play a key role in informing treatment pathways and contribute to the debate on specific treatments (e.g., cannabinoids). A group of expert clinicians (pain specialists and psychiatrists), scientists, and patient representatives convened to assess the relative benefit-safety balance of 12 pharmacological treatments, including orally administered cannabinoids/cannabis-based medicinal products, for the treatment of CNP in adults. Methods: A decision conference provided the process of creating a multicriteria decision analysis (MCDA) model, in which the group collectively scored the drugs on 17 effect criteria relevant to benefits and safety and then weighted the criteria for their clinical relevance. Findings: Cannabis-based medicinal products consisting of tetrahydrocannabinol/cannabidiol (THC/CBD), in a 1:1 ratio, achieved the highest overall score, 79 (out of 100), followed by CBD dominant at 75, then THC dominant at 72. Duloxetine and the gabapentinoids scored in the 60s, amitriptyline, tramadol, and ibuprofen in the 50s, methadone and oxycodone in the 40s, and morphine and fentanyl in the 30s. Sensitivity analyses showed that even if the pain reduction and quality-of-life scores for THC/CBD and THC are halved, their benefit-safety balances remain better than those of the noncannabinoid drugs. Interpretation: The benefit-safety profiles for cannabinoids were higher than for other commonly used medications for CNP largely because they contribute more to quality of life and have a more favorable side effect profile. The results also reflect the shortcomings of alternative pharmacological treatments with respect to safety and mitigation of neuropathic pain symptoms. Further high-quality clinical trials and systematic comprehensive capture of clinical experience with cannabinoids is warranted. These results demonstrate once again the complexity and multimodal mechanisms underlying the clinical experience and impact of chronic pain.

Advancing structured decision-making in drug regulation at the FDA and EMA.

The recent benefit-risk framework (BRF) developed by the Food and Drug Administration (FDA) is intended to improve the clarity and consistency in communicating the reasoning behind the FDA's decisions, acting as an important advancement in US drug regulation. In the PDUFA VI implementation plan, the FDA states that it will continue to explore more structured or quantitative decision analysis approaches; however, it restricts their use within the current BRF that is purely qualitative. By contrast, European regulators and researchers have been long exploring the use of quantitative decision analysis approaches for evaluating drug benefit-risk balance. In this paper, we show how quantitative modelling, backed by decision theory, could complement and extend the FDA's BRF to better support the appraisal of evidence and improve decision outcomes. After providing relevant scientific definitions for benefit-risk assessment and describing the FDA and European Medicines Agency (EMA) frameworks, we explain the components of and differences between qualitative and quantitative approaches. We present lessons learned from the EMA experience with the use of quantitative modelling and we provide evidence of its benefits, illustrated by a real case study that helped to resolve differences of judgements among EMA regulators.

So near yet so far: why won't the UK prescribe medical cannabis?

Although cannabis-based products for medicinal use are now legal in the UK, it is still challenging for patients to gain access, and only very few National Health Service prescriptions have been written to date. This paper attempts to make sense of why the UK lags behind so many other countries which also have legalised medical cannabis. From consulting with parents and patients, prescribers, pharmacists and decision-makers it seems that there are a series of distinct barriers to prescribing that need to be overcome in order to improve patient access to medical cannabis in the UK. These include concerns about the perceived lack of scientific evidence. To alleviate these concerns, we highlight the importance of patient-centred approaches including patient-reported outcomes, pharmacoepidemiology and n=1 trials, which can contribute to the development of the evidence base for medical cannabis. We hope that this paper will help policymakers and prescribers understand the challenges to prescribing and so help them develop approaches to overcome the current situation which is detrimental to patients.

ICER Value Framework 2020 Update: Recommendations on the Aggregation of Benefits and Contextual Considerations.

The Institute for Clinical and Economic Review (ICER) in the United States recently published a 2020 update to its value assessment framework. We are commenting on the method by which the benefits of health interventions are integrated, relating to contextual considerations and other factors relevant to an intervention's value. We start by discussing the theoretical foundations of decision analysis and its extension to multiple criteria decision analysis (MCDA). Then we provide a detailed, evidence-based response to some of the claims made by ICER with regard to the use of MCDA methods and stakeholder engagement. Finally, we provide a number of recommendations on the use of quantitative decision analysis and decision conferencing that could be of relevance to the ICER methodology. Overall, we agree that some of the proposed changes by ICER are moving in the right direction toward improving transparency in the value assessment process, but these changes are probably inadequate. We advocate that more serious attention should be paid to the use of quantitative decision analysis together with decision conferencing for the construction of value preferences via group processes for the integration of an intervention's various benefit components.

Multi-Criteria Decision Analysis to Develop an Efficacy-Safety Profile of Parenteral Analgesics Used in the Treatment of Postoperative Pain.

BACKGROUND: Identifying the optimal treatment in an acute postoperative setting remains a challenge. Multiple analgesic options are available, but comparing outcomes is limited by a lack of head-to-head trials. In addition, decisions based on efficacy only do not take drug safety into account. In such cases, multi-criteria decision analysis (MCDA) can be utilized to quantify and compare the efficacy and safety data of various drugs. METHODOLOGY: The efficacy-safety profiles of eight parenteral, postoperative analgesics (acetaminophen, diclofenac, ketorolac, metamizole, morphine, nefopam, parecoxib, tramadol) widely used in Europe were evaluated using an MCDA model that included 17 criteria: three for efficacy and 14 for safety. Each drug was scored on each criterion on a scale from 0 (worst) to 100 (best), according to published data and the judgment of an expert panel. A weighting process was then applied to standardize the impact of each criterion and adjust drugs' preference scores accordingly, normalizing them on the 0-100 scale. Sensitivity analyses were also performed, including a model in which analgesic profiles were compared when opioid sparing effect was set at a zero value for all drugs. RESULTS: In the primary model, efficacy and safety had relative weightings of 64% and 36%, respectively. Efficacy and safety criteria with the highest values were pain relief (relative weight, 29%) and gastrointestinal effects (12%). Parecoxib received the highest overall score (93), followed by diclofenac (80), and ketorolac (75). Morphine scored the lowest (57), due to the lack of an opioid sparing effect. When opioid sparing was given a zero rating, parecoxib remained the highest scoring analgesic (93), followed by diclofenac (80), metamizole (76), and morphine (76). Parecoxib remained the most preferred analgesic in several other sensitivity analyses. CONCLUSION: This MCDA-based assessment suggests that parecoxib has the most favorable efficacy-safety profile among the assessed postoperative analgesics.

Medical cannabis in the UK: From principle to practice.

BACKGROUND: In the UK, medical cannabis was approved in November 2018, leading many patients to believe that the medicine would now be available on the NHS. Yet, to date, there have been only 12 NHS prescriptions and less than 60 prescriptions in total. In marked contrast, a recent patient survey by the Centre for Medical Cannabis (Couch, 2020) found 1.4 m people are using illicit cannabis for medical problems. AIMS: Such a mismatch between demand and supply is rare in medicine. This article outlines some of the current controversies about medical cannabis that underpin this disparity, beginning by contrasting current medical evidence from research studies with patient-reported outcomes. OUTCOMES: Although definite scientific evidence is scarce for most conditions, there is significant patient demand for access to medical cannabis. This disparity poses a challenge for prescribers, and there are many concerns of physicians when deciding if, and how, to prescribe medical cannabis which still need to be addressed. Potential solutions are outlined as to how the medical profession and regulators could respond to the strong demand from patients and families for access to medical cannabis to treat chronic illnesses when there is often a limited scientific evidence base on whether and how to use it in many of these conditions. CONCLUSIONS: There is a need to maximise both clinical research and patient benefit, in a safe, cautious and ethical manner, so that those patients for whom cannabis is shown to be effective can access it. We hope our discussion and outlines for future progress offer a contribution to this process.

Multicriteria Decision Analysis Applied to the Clinical Use of Pharmacotherapy for Overactive Bladder Symptom Complex.

CONTEXT: The nonspecific storage symptom complex overactive bladder (OAB) is an important clinical condition in functional urology. Until recently, pharmacological therapy comprised antimuscarinic drugs, but more recently beta 3 agonists have added to the available agents. Traditional reporting of efficacy and safety of these agents relies upon regulatory placebo-controlled studies. There remains no head-to-head comparison of existing agents in the contemporary literature. Contemporary conclusions on comparative efficacy and safety drawn from the use of these agents are based on systematic reviews of the literature and associated meta-analyses. OBJECTIVE: In this study, we used the analytical model of multicriteria decision analysis (MCDA) to compare contemporary pharmacotherapy for OAB. EVIDENCE ACQUISITION: Efficacy and safety data from published, randomised, placebo-controlled trials of antimuscarinic antagonists, the beta 3 agonist, and the combination of an antimuscarinic and beta 3 agonist were used to populate the MCDA model. EVIDENCE SYNTHESIS: Experts assessed weights of the relative importance of favourable and unfavourable effects, which provided a common measure of benefits and safety that were combined in the MCDA model to give an overall ranking of the OAB drugs. RESULTS: When benefits are judged as more important than safety, fesoterodine 4 or 8mg used in a flexible dosing pattern provides the most favourable therapeutic option, over a wide sensitivity analysis relating to benefits and harms. CONCLUSIONS: In our analysis using an MCDA model, in both the flexible dosing pattern of fesoterodine and the solifenacin combination with mirabegron, the benefit-safety balance is better in terms of benefits and/or safety than any of the other available OAB drugs. Caution in interpretation of the data has to be expressed as the fesoterodine data are based on a flexible dosing regimen, which adds an additional dimension of personalising therapy. PATIENT SUMMARY: Overactive bladder (OAB) is a common condition with a significant impact on the quality of life. Possible symptoms include the following: (1) urgency-a compelling desire to urinate, which is difficult to defer; (2) urgency urinary incontinence-urgency leading to incontinence episodes; (3) frequency-increased frequency of wanting to pass urine; and (4) nocturia-increase in instances of getting up at night to urinate. To date, the mainstay of therapy for OAB has been antimuscarinic drugs and, more recently, the beta 3 agonist mirabegron. Ten international experts in urology, obstetrics, gynaecology, healthy ageing, and data analysis compared the benefit-risk balance of 14 OAB drugs licensed in Europe. The experts considered the importance of a favourable effect on the above four symptoms and also potential for side effects, but only three of these side effects, constipation, dry mouth, and dizziness, showed clinically relevant differences among the six drugs they considered. The observations recorded here suggest interesting differences between drugs across a wide range of possible trade-offs between benefit and safety. The different recruitment criteria used for each study may influence the results seen, so they need to be treated with caution. Comparison of flexibly dosed fesoterodine studies with fixed-dose fesoterodine studies introduces an additional potential bias; definitive conclusions can be drawn only if enough comparable placebo-controlled flexible dosing studies with other drugs were available.

Quantifying Preferences in Drug Benefit-Risk Decisions.

Benefit-risk assessment is used in various phases along the drug lifecycle, such as marketing authorization and surveillance, health technology assessment (HTA), and clinical decisions, to understand whether, and for which patients, a drug has a favorable or more valuable profile with reference to one or more comparators. Such assessments are inherently preference-based as several clinical and nonclinical outcomes of varying importance might act as evaluation criteria, and decision makers must establish acceptable trade-offs between these outcomes. Different healthcare stakeholder perspectives, such as those from patients and healthcare professionals, are key for informing benefit-risk trade-offs. However, the degree to which such preferences inform the decision is often unclear as formal preference-based evaluation frameworks are generally not used for regulatory decisions, and, if used, rarely communicated in HTA decisions. We argue that for better decisions, as well as for reasons of transparency, preferences in benefit-risk decisions should more often be quantified and communicated explicitly.

A new approach to formulating and appraising drug policy: A multi-criterion decision analysis applied to alcohol and cannabis regulation.

BACKGROUND: Drug policy, whether for legal or illegal substances, is a controversial field that encompasses many complex issues. Policies can have effects on a myriad of outcomes and stakeholders differ in the outcomes they consider and value, while relevant knowledge on policy effects is dispersed across multiple research disciplines making integrated judgements difficult. METHODS: Experts on drug harms, addiction, criminology and drug policy were invited to a decision conference to develop a multi-criterion decision analysis (MCDA) model for appraising alternative regulatory regimes. Participants collectively defined regulatory regimes and identified outcome criteria reflecting ethical and normative concerns. For cannabis and alcohol separately, participants evaluated each regulatory regime on each criterion and weighted the criteria to provide summary scores for comparing different regimes. RESULTS: Four generic regulatory regimes were defined: absolute prohibition, decriminalisation, state control and free market. Participants also identified 27 relevant criteria which were organised into seven thematically related clusters. State control was the preferred regime for both alcohol and cannabis. The ranking of the regimes was robust to variations in the criterion-specific weights. CONCLUSION: The MCDA process allowed the participants to deconstruct complex drug policy issues into a set of simpler judgements that led to consensus about the results.

Selecting pH cut-offs for the safe verification of nasogastric feeding tube placement: a decision analytical modelling approach.

OBJECTIVES: The existing British National Patient Safety Agency (NPSA) safety guideline recommends testing the pH of nasogastric (NG) tube aspirates. Feeding is considered safe if a pH of 5.5 or lower has been observed; otherwise chest X-rays are recommended. Our previous research found that at 5.5, the pH test lacks sensitivity towards oesophageal placements, a major risk identified by feeding experts. The aim of this research is to use a decision analytic modelling approach to systematically assess the safety of the pH test under cut-offs 1-9. MATERIALS AND METHODS: We mapped out the care pathway according to the existing safety guideline where the pH test is used as a first-line test, followed by chest x-rays. Decision outcomes were scored on a 0-100 scale in terms of safety. Sensitivities and specificities of the pH test at each cut-off were extracted from our previous research. Aggregating outcome scores and probabilities resulted in weighted scores which enabled an analysis of the relative safety of the checking procedure under various pH cut-offs. RESULTS: The pH test was the safest under cut-off 5 when there was ≥30% of NG tube misplacements. Under cut-off 5, respiratory feeding was excluded; oesophageal feeding was kept to a minimum to balance the need of chest X-rays for patients with a pH higher than 5. Routine chest X-rays were less safe than the pH test while to feed all without safety checks was the most risky. DISCUSSION: The safety of the current checking procedure is sensitive to the choice of pH cut-offs, the impact of feeding delays, the accuracy of the pH in the oesophagus, as well as the extent of tube misplacements. CONCLUSIONS: The pH test with 5 as the cut-off was the safest overall. It is important to understand the local clinical environment so that appropriate choice of pH cut-offs can be made to maximise safety and to minimise the use of chest X-rays. TRIAL REGISTRATION NUMBER: ISRCTN11170249; Pre-results.

Recommendations for benefit-risk assessment methodologies and visual representations.

PURPOSE: The purpose of this study is to draw on the practical experience from the PROTECT BR case studies and make recommendations regarding the application of a number of methodologies and visual representations for benefit-risk assessment. METHODS: Eight case studies based on the benefit-risk balance of real medicines were used to test various methodologies that had been identified from the literature as having potential applications in benefit-risk assessment. Recommendations were drawn up based on the results of the case studies. RESULTS: A general pathway through the case studies was evident, with various classes of methodologies having roles to play at different stages. Descriptive and quantitative frameworks were widely used throughout to structure problems, with other methods such as metrics, estimation techniques and elicitation techniques providing ways to incorporate technical or numerical data from various sources. Similarly, tree diagrams and effects tables were universally adopted, with other visualisations available to suit specific methodologies or tasks as required. Every assessment was found to follow five broad stages: (i) Planning, (ii) Evidence gathering and data preparation, (iii) Analysis, (iv) Exploration and (v) Conclusion and dissemination. CONCLUSIONS: Adopting formal, structured approaches to benefit-risk assessment was feasible in real-world problems and facilitated clear, transparent decision-making. Prior to this work, no extensive practical application and appraisal of methodologies had been conducted using real-world case examples, leaving users with limited knowledge of their usefulness in the real world. The practical guidance provided here takes us one step closer to a harmonised approach to benefit-risk assessment from multiple perspectives.

Literature review of visual representation of the results of benefit-risk assessments of medicinal products.

BACKGROUND: The PROTECT Benefit-Risk group is dedicated to research in methods for continuous benefit-risk monitoring of medicines, including the presentation of the results, with a particular emphasis on graphical methods. METHODS: A comprehensive review was performed to identify visuals used for medical risk and benefit-risk communication. The identified visual displays were grouped into visual types, and each visual type was appraised based on five criteria: intended audience, intended message, knowledge required to understand the visual, unintentional messages that may be derived from the visual and missing information that may be needed to understand the visual. RESULTS: Sixty-six examples of visual formats were identified from the literature and classified into 14 visual types. We found that there is not one single visual format that is consistently superior to others for the communication of benefit-risk information. In addition, we found that most of the drawbacks found in the visual formats could be considered general to visual communication, although some appear more relevant to specific formats and should be considered when creating visuals for different audiences depending on the exact message to be communicated. CONCLUSION: We have arrived at recommendations for the use of visual displays for benefit-risk communication. The recommendation refers to the creation of visuals. We outline four criteria to determine audience-visual compatibility and consider these to be a key task in creating any visual. Next we propose specific visual formats of interest, to be explored further for their ability to address nine different types of benefit-risk analysis information.

Balancing benefit and risk of medicines: a systematic review and classification of available methodologies.

BACKGROUND: The need for formal and structured approaches for benefit-risk assessment of medicines is increasing, as is the complexity of the scientific questions addressed before making decisions on the benefit-risk balance of medicines. We systematically collected, appraised and classified available benefit-risk methodologies to facilitate and inform their future use. METHODS: A systematic review of publications identified benefit-risk assessment methodologies. Methodologies were appraised on their fundamental principles, features, graphical representations, assessability and accessibility. We created a taxonomy of methodologies to facilitate understanding and choice. RESULTS: We identified 49 methodologies, critically appraised and classified them into four categories: frameworks, metrics, estimation techniques and utility survey techniques. Eight frameworks describe qualitative steps in benefit-risk assessment and eight quantify benefit-risk balance. Nine metric indices include threshold indices to measure either benefit or risk; health indices measure quality-of-life over time; and trade-off indices integrate benefits and risks. Six estimation techniques support benefit-risk modelling and evidence synthesis. Four utility survey techniques elicit robust value preferences from relevant stakeholders to the benefit-risk decisions. CONCLUSIONS: Methodologies to help benefit-risk assessments of medicines are diverse and each is associated with different limitations and strengths. There is not a 'one-size-fits-all' method, and a combination of methods may be needed for each benefit-risk assessment. The taxonomy introduced herein may guide choice of adequate methodologies. Finally, we recommend 13 of 49 methodologies for further appraisal for use in the real-life benefit-risk assessment of medicines.

Estimating the harms of nicotine-containing products using the MCDA approach.

BACKGROUND: An international expert panel convened by the Independent Scientific Committee on Drugs developed a multi-criteria decision analysis model of the relative importance of different types of harm related to the use of nicotine-containing products. METHOD: The group defined 12 products and 14 harm criteria. Seven criteria represented harms to the user, and the other seven indicated harms to others. The group scored all the products on each criterion for their average harm worldwide using a scale with 100 defined as the most harmful product on a given criterion, and a score of zero defined as no harm. The group also assessed relative weights for all the criteria to indicate their relative importance. FINDINGS: Weighted averages of the scores provided a single, overall score for each product. Cigarettes (overall weighted score of 100) emerged as the most harmful product, with small cigars in second place (overall weighted score of 64). After a substantial gap to the third-place product, pipes (scoring 21), all remaining products scored 15 points or less. INTERPRETATION: Cigarettes are the nicotine product causing by far the most harm to users and others in the world today. Attempts to switch to non-combusted sources of nicotine should be encouraged as the harms from these products are much lower.

Structured Approaches to Benefit-Risk Assessment: A Case Study and the Patient Perspective.

Assessing the utility of structured approaches to benefit-risk assessment of medicinal products is challenging, in part due to the lack of a gold standard for results and the uncertainty inherent in the data. In place of conducting formal testing, obtaining feedback from users of structured approaches provides insight into their value and limitations. The authors conducted a simulated single-session benefit-risk decision in which 3 groups applied the PhRMA BRAT(Pharmaceutical Research and Manufacturers of America Benefit-Risk Action Team) framework or the multicriteria decision analysis approach. The groups were provided with background and data for a hypothetical triptan for acute migraine in a population with cardiovascular risk factors and were asked to determine and defend an approval decision. Three insights emerged consistently from the groups: (1) the value of a structured approach to benefit-risk assessment, (2) the clarity provided by real-time visualization tools, and, most critically, (3) the importance of bringing the patient into the discussion early.

Risk perception of prescription drugs: results of a survey among experts in the European regulatory network.

BACKGROUND: Experts are perceived to be veridical and to focus only on objective data when evaluating risk. Only a few research studies have attempted to characterize the subjectivity in risk evaluation among experts. OBJECTIVE: The hypothesis of this study is that expert evaluation of a pharmaceutical drug can be partly explained by dimensions that describe the drug and by individual characteristics. DESIGN: Seventy-five medical assessors in 9 EU countries evaluated a list of 28 pharmaceutical drugs using 4 scales: risk, benefit, seriousness of harm, and patients' knowledge of the risk. They were also given a mock "clinical dossier" and asked to rate it on 8 dimensions: risk, benefit, worry, magnitude of the exposure, scientific knowledge of the risk, familiarity of the risk, ethical concerns, and risk acceptability. RESULTS: Female assessors perceived significantly higher benefits than men for a large number of the 28 drugs. Principal component analysis of the ratings for the clinical dossiers revealed 2 underlying components: seriousness of harm and scientific evidence. A regression model predicting the risk perception of the drug showed that the variables seriousness of harm (benefit, worry, magnitude of exposure, ethical concerns, and risk acceptability), years of regulatory experience, gender, and type of drug explained 54% of the variability among assessors. CONCLUSION: Assessors' view of the risks associated with pharmaceutical drugs is influenced by worry for patient safety, magnitude of patient exposure, and ethical concerns. These dimensions may influence their perceptions of benefit and risk acceptability. Senior assessors are more risk averse than junior assessors, and female assessors seem to be sensitive to the promise of benefit from medicines and consequently may be less risk averse than male assessors.